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Abstract 

This paper examines a range of methods to collect data for the investigation of 
decision-making in airline Operations Control Centres (OCCs). A study was 
conducted of 52 controllers in five OCCs of both domestic and international 
airlines in the Asia-Pacific region. A range of methods was used including: 
surveys, interviews, observations, simulations, and think-aloud protocol. The 
paper compares and evaluates the suitability of these techniques for gathering data 
and provides recommendations on the application of simulations. 
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In airline operations, decisions yielding optimal results may be the difference between 
company survival and bankruptcy. Stakes are high, timing is crucial, and the consequences 
of poor decisions could be critical. An airline’s Operations Control Centre (OCC) operates 
in a complex, dynamic, and intense environment and serves as the airline’s nerve centre. It 
is responsible for the control of aircraft to ensure economical, operational (Williams, 1967) 
and commercial efficiency. Although decision-making has been examined in the aviation 
industry, for example in Air Traffic Control (Corker, Pisanich and Bunzo, 1997), Pilot 
Crewing (Fischer and Orasanu, 1997), Schedule Disruption Management (Thengvall, Bard 
and Yu, 2000), and in Emergency Services (Flin, Stewart, and Slaven, 1996), empirical 
research on OCC decision-making is very limited. Understanding decision-making in 
OCCs requires the investigator to gain an appreciation of the demands presented by 
disruptions to operational schedules and the necessity to recover the situations, often under 
severe time constraints. A study of decision-making would require an investigator to be 
present 24 hours a day for many days, or even weeks in the international environment, to 
gain a full understanding of the work demands. Alternatively, an investigation would rely 
on the abilities of personnel to recall events and responses well after the disruption, 
exposing the investigation to inaccuracies. Consequently, studying decision-making in 
OCCs presents many challenges in terms of the study design. This paper examines and 
evaluates a range of methods for gathering data in airline operations. 

Review of Data Collection Methods 

Investigators face a difficult task in attempting to capture data accurately in a way that is 
methodologically sound so that generalisations can be made (Bordia and Rosnow, 1998). 
According to Zikmund (2000), the main primary data collection methods include surveys 
(including questionnaires and interviews), observation, and experiments. This paper 
examines the appropriateness of each of these methods for investigating decision-making in 
OCCs. 

Survey 

A common method of collecting data is the use of a survey (Sekaran, 1992) based on pre- 
determined questions. A survey can be widely distributed at reasonable cost and is an 
excellent basis for describing people’s attitudes and behaviours (Shaughnessy and 
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Zechmeister, 1994). However gathering information which relies on the recall of past 
events and behaviour may be of dubious validity and reliability (Foddy, 1994). Further, the 
current study was predicated on self-reflection by controllers and accurate analysis of 
events often well after the events occurred. There could be a lack of accuracy in recalling 
events, and confusion of details particularly where simultaneous disruptions occurred. A 
major weakness of surveys is that they are ill-suited for in-depth examination of the 
thoughts and feelings of individual respondents (Shaughnessy and Zechmeister, 1994). In 
addition, surveys do not provide the means to explore uncertainty or to clarify responses 
(Sekaran, 1992). 

Interviews 

The use of interviews as a method of data collection provides a means for establishing 
rapport with respondents, and exploring and understanding complex issues (Sekaran, 
1992). Interviews allow researchers to adapt the questions as necessary, clarify doubts, and 
ensure responses are understood properly (Sekeran, 1992). Interviews also assist in 
understanding how 7 individuals perceive and give meaning to cer tain phenomena or events 
(Berg, 1995). For example Yorkston, Klasner and Swanson (2001) used in-depth 
interviews to gain insider perspectives of communication by individuals (N=7) with 
multiple sclerosis. Yorkston et al (2001) found that this process of interviewing 
respondents yielded rich information. Interviews, therefore, may be useful for studying 
decision-making by controllers in OCCs as they allow a great deal of interaction between 
researcher and respondent. However there are limitations with interviews. Yin (1994) 
warns that interviews are verbal reports only, and are thus subject to bias, poor recall, and 
poor or inaccurate articulation. To address this issue in the current study interviews could 
be conducted during disruptions to elicit accurate and timely information but such action 
could interfere with the work process and controllers could be unwilling to cooperate. 
Even if interviews are conducted at a later time, the problems of accurate recall of details 
would still be a problem. 

Observation 

Observation in the work setting is useful to further understanding of the context or 
phenomenon (Yin, 1994) and establishes the validity of the findings. Observation enables 
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researchers to distinguish between real and verbal behaviour (Bogdewic, 1999), and gather 
data at the time behaviour occurs (Atherton and Klemmack, 1982). In OCCs, observing 
disruptions as they occur permits the researcher to monitor the progress of the disruption 
and any actions being taken by controllers. Boote and Matthews (1999) found 
observation to be the most appropriate research methodology to observe customer traffic 
flows along main streets. Observational research requires no effort on the part of the 
respondent, and is a very effective way to enrich and supplement data gathered by other 
means (Boote and Matthews, 1999). However, attitudes and opinions cannot be recorded 
using observation (Zikmund, 2000). Systematic observation may be costly and very time 
consuming, and the lack of predictability of events may also create problems for 
researchers. For example in OCCs, observation may be conducted over several hours or 
even days without incident. Further, Zikmund (2000) noted several deficiencies with the 
collection of data using this technique such as observer subjectivity, memory limitations, 
inability to record all details, and inability to interpret observations such as body language, 
appropriately. Consequently in the current study observation alone would not capture the 
required information. 

Think-aloud protocol 

Think-aloud protocol is a technique for verbalisation, where participants are instructed to 
think aloud as they work on a problem (Woods, 1993). This technique allows the 
researcher to access the underlying thought processes, reasoning, and behaviours involved 
in analysing and solving the problem (Ericsson and Simon, 1993). Respondents in 
numerous studies have been asked to think aloud and talk through their thoughts. For 
example de Groot (1978) presented chess players (N=22) with various combinations of 
chess pieces on a board, and asked the players to think aloud while selecting the best move. 
De Groot (1978) noted that most subjects found thinking aloud, slowed thinking down, and 
made the subject think more explicitly. Think-aloud protocol may be appropriate to gain 
an insight of controllers’ thought processes in decision-making. However, controllers do 
not have time during actual disruptions to elucidate their reasons for taking particular 
actions. 

Methods for recording responses 
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Various methods and devices can be used to record responses, such as tape recordings of 
dialogue, written responses, computer inputs, observations, or by some combination of 
these approaches. However the method used may influence the results. For example 
having participants respond in writing may be tedious and limiting. Adelman, Tolcott and 
Bresnick (1993) examined the effect of gathering data on quality judgement with trained 
Army Air Defence personnel (N=63), and concluded that die artificialities of the paper and 
pencil task may have influenced the results. 

Case study approach 

The case study method is well suited to exploratory or descriptive studies (Yin, 1994). The 
multiple case study approach has the benefits of deepening the understanding of the subject 
(Miles and Huberman, 1994), enabling more information to be gathered and increasing the 
generalisability of the research (Bryman, 1989). Using a case study method would allow 
the investigation of decision-making in several OCCs. The case study method may be 
limited due to the necessity to gain access to (Burgess, 1984), and cooperation of (Zikmund 
2000) a number of suitable companies, and to the time taken to complete die study (Yin, 
1984). 

Experiments and simulations 

In an experiment, variables may be manipulated in controlled conditions to test a 
hypothesis (Zikmund, 2000). Often comparisons are made between results from a control 
group and an experimental group (Burgess, 1993). A simulation is akin to an experiment; 
specially created in an artificial setting, but not very different from reality (Sekaran, 1992). 
One of the most important aspects of simulation is its ability to mimic reality' so that 
researchers can study phenomena too difficult or inpossible to study in real life (Bordt, 
1999). The use of simulation is also recognised as one of the most widely used techniques 
in operational research and management science (Law and Kelton, 1991). The ability to 
control the simulation allows the replication of situations, and the manipulation of certain 
conditions (Blank, 1984). These aspects were important considerations in the current 
study. 
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Simulations have wide applications in research and practice. Simulation has been used as a 
decision support tool for management. For example, in a study of operational changes in 
an army hospital, simulation was used to assess alternative patient capacity and staffing 
capabilities (Ledlow and Bradshaw, 1999). Simulation may also be used in a teaching or 
training context. According to Salas and Burke (2002), simulation may be effective 
provided that it is instructional, meets task, performance, and feedback needs, and helps to 
guide learning. Bordt (1999) found simulation to be a very effective teaching aid to 
students studying criminology, because the students became fully engaged in the process, 
could readily test real life situations, and demonstrated critical evaluation skills. Fisher, 
Laurie, Glaser, Connemey, Pollatsek, Duffys and Brock (2002) used simulation to test a 
PC-based driving skills program with young drivers (N=45). Simulation provided a safe 
environment to study the effects of experience and risk awareness on drivers. 

In aviation, one of the most useful applications of simulation is the flight simulator which 
allows the representation of aircraft flight with safety, training, and financial efficiencies 
(Moroney and Moroney, 1999). Simulation is also used extensively in Air Traffic Control, 
particularly for selection and training (Ackerman and Kanfer, 1993). The conclusion from 
the use of simulation in these domains is that where appropriately designed and used, 
simulation can be a most effective tool for testing, .teaching, and learning. The suitability 
of simulations as a data collection method relies on the degree to which the simulation 
replicates the work situation. . Therefore it is important to pay attention to detail in the 
design of simulations. 

In summary, all the methods of data collection considered have limitations. Therefore, 
collecting data through multi-methods, although costly and time-consuming, lends rigour to 
research (Sekaran, 1992). The current study combined a number of data-collection 
methods to counter the deficiencies of any single method. This paper focuses on data 
collection methods and therefore the substantive findings are not reported in the paper. 

Data collection in the current study 

A survey was distributed as a preliminary test to controllers (n=6) from one Australian 
domestic OCC. Respondents were asked to complete a questionnaire at the conclusion of 
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each disruption or at the end of each shift. Questions asked respondents to record the 
nature, cause and duration of each disruption. Respondents were asked to answer questions 
such as ‘where did they obtain information for decision-making’, and ‘what amount of 
information did they need to make decisions’? Qualitative data were also collected by 
observing controllers during disruptions and asking unstructured and semi-structured 
questions. 

However the use of surveys and observations to collect data about decision- making during 
disruptions was inappropriate. The frequency, regularity, and duration of disruptions were 
unpredictable, making data collection inefficient. Some disruptions continued across a 
number of shifts or even days. Therefore, decision-making processes of any one 
respondent were difficult to capture. Second, respondents were generally unable or 
unwilling to answer questions during a disruption as they needed to focus intensely on 
solving operational problems. Third, there was no way to compare the decision-making 
outcomes of respondents. Some disruptions were resolved quickly and with minimal 
decision-making from controllers, while others were longer and required extensive 
decision-making. Fourth, if a number of disruptions occurred simultaneously, decision- 
making relevant to any particular disruption could not necessarily be determined. Fifth, 
there was no means by which the researcher could be confident that respondents completed 
the questionnaire with any degree of accuracy or timeliness. The conclusion reached was 

that the method using surveys and observation to study decision-making in an OCC was 
inappropriate. 

Simulation study 

The study was re-designed using a multiple case study approach to examine decision 
outcomes of controllers (N=52) in five Operations Control Centres. The study was 
broadened to include OCCs in other Australian domestic airlines, as well as OCCs of 
international airlines in the Asia-Pacific region. One airline operated solely domestic 
services, two operated solely international services and two operated both domestic and 
international services. In domestic operations, flight stages are short, aircraft are scheduled 
to operate a high number of flight stages in a day, and are on the ground for short durations 
between flights. Controllers handling domestic disruptions, therefore, make numerous 
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decisions within very short timeframes. In contrast, international flight stages may be very 
long, aircraft may only operate a small number of flights per day, and are on the ground 
between flights for longer durations. Decision-making therefore is generally less intense. 

A simulation to replicate each of the operational environments was designed in 
collaboration with a panel of experts. The experts were current or retired senior operations 
managers (N=10). For each simulation, hypothetical flight schedules were constructed to 
replicate typical airline schedules. Each simulation consisted of a visual display showing a 
coloured utilisation of a fleet of aircraft, and was laminated to allow respondents to write or 
draw on the surface. The display was a close representation of the computerised display 
normally used by operations controllers. Information on the display included aircraft flight 
stages with flight numbers, city pairs (routes), flight loadings, maintenance 
unserviceabilities and requirements, and other relevant information. The flight stages were 
positioned on the display according to a continuous time scale. A time-line could be 
positioned along the time-scale to indicate the specific time of day, or a passage of elapsed 
time. 

Domestic Simulation 

The domestic simulation was designed for OCCs in the Australasian region. Hence the 
flight schedule was constructed for a hypothetical airline operating within Australia. The 
base location of the airline was irrelevant for the study. 

International Simulation 

The international simulation was designed for OCCs in the Asia-Pacific region. Hence the 
hypothetical airline was based at a fictitious location, but within the Pacific region. 
Information for respondents included the approximate location depicted on a map, and the 
local time zone of the location. Respondents could determine the flying time and therefore 
distance to any destination of the airline from the flight schedules. 

A series of simulation scenarios was designed in collaboration with the panel of experts. 
The scenarios consisted of three typical operational problems for each of the domestic and 
international operations, and each successive scenario was more complex than the previous 
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one, in terms of the nature of the disruption, the consequences of any decision, and 
timeframes for decision-making. The three scenarios were completely independent of each 
other. A short audio briefing tape was made for each of the domestic and international 
simulations. The tape served to ensure a consistent briefing for each respondent, and 
contained important information for the simulation. The preamble outlined the reason for 
the study and emphasised to the respondent that the study was concerned with decision- 
making processes rather than solutions to the scenarios. Respondents were asked to think 
aloud as the simulation proceeded. 

The simulation was conducted between the researcher and one respondent at a time at the 
respondent’s workplace, but in a separate room. Respondents were invited by their 
managers to participate voluntarily in the study. As only three OCC controllers from three 
different airlines declined the study, the sample was highly representative of the population 
of controllers in the airlines studied. While large sample sizes are more reliable, they are 
also more costly (Blank, 1984). The study was limited to the Asia-Pacific region for this 
reason. In the OCCs of the two airlines in the study that operated both domestic and 
international operations, respondents worked either as domestic or international controllers. 
These respondents were permitted to select either die domestic or international simulation. 

Prior to running the simulation, each respondent listened to die five minute audio briefing 
tape. Respondents were asked if any information required clarification or whether they felt 
they needed additional information. When respondents indicated their readiness to 
commence the simulation, the tape recorder was set to record all subsequent 
communications. Taping is more accurate than any other method of recording interview 
data, providing the respondent agrees to being taped, and the taped data are transcribed 
(Yin, 1994). The current study satisfied both these criteria. The taping process did not 
appear to present any difficulties and the respondents did not react adversely to the tape 
recorder, even to the extent of waiting while tapes were changed. According to Ericsson 
and Simon, (1993) respondents become so involved in the task that they soon become 
accustomed to the presence of the taping device. 
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The researcher commenced the simulation by showing respondents the utilisation of flight 
schedules and describing the display. The respondents were asked to familiarise 
themselves with the utilisation, and communicate observations and any questions to the 
researcher. As each scenario was introduced, respondents were asked to think aloud as 
they attempted to solve the operational problems. The researcher played two critical roles 
throughout the simulation. The main role of the researcher during the simulation was to 
control and direct each scenario according to a script. As the scenario progressed, further 
information was given to respondents at appropriate times. Typical developments included 
sudden maintenance failures, deteriorating weather conditions, or particular commercial 
requirements. Second, the researcher acted as a resource base for the scenario, providing 
any information required by respondents as the scenario progressed. On request from 
respondents, the researcher provided information or answered questions relating, for 
example, to crew availability or roster commitments, passenger connections, airport or air 
traffic control related information, maintenance requirements, and weather advice. 

As each scenario progressed, respondents were required to make decisions and attempt to 
construct solutions to the operational problems. By observing respondents’ actions, and 
listening to their verbalised thought processes, the researcher could explore the decision- 
making process. For example if a respondent noted several flights with low loadings 
(proportion of booked passengers, to available seats), the researcher could probe the reason 
for the respondent’s comment. The researcher also asked questions such as ‘why do you 
doubt the information provided by Engineering?’ or ‘how do you know when you have 
enough information to help you make a decision?’ The scenario was continued until 
respondents considered a problem was solved satisfactorily or the researcher considered 
little would be gained by extending the scenario. 

Discussion 

Most respondents were familiar with details contained within the simulated displays. 
However, respondents working in OCCs outside Australia, but participating in the domestic 
simulation, required clarification of information such as Australian airport abbreviations. 
Respondents were asked to treat the initial familiarisation of schedules as akin to 
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commencing a normal shift. The researcher provided a briefing (handover) to facilitate this 
process. 

Respondents described their thought processes throughout the simulation and readily 
offered comments or answered questions as the scenarios progressed. Periods of silence 
did not su gg est disinterest or loss of awareness of the scenario. During these periods, the 
researcher prompted respondents to verbalise their thoughts by asking questions such as 
‘what are you looking at’? or ‘why are you indicating this flight’? Respondents became 
totally absorbed in the scenarios. At times the researcher had to clarify flight numbers or 
aircraft identifications to ensure the tapes could be transcribed accurately. 

Preliminary findings 

The influence of respondents’ experience in other areas of the airline became apparent in 
their approach to tackling the task. For example, respondents who had a crew rostering 
background tended to give priority to crew-related problems during the scenarios. 
Similarly respondents who had experience in airline ground-handling roles, attended 
initially to issues such as airport congestion or potential aircraft handling problems. These 
preliminary findings suggest that a controller’s previous experience, particularly in areas 
outside OCCs may influence the decision-making process. The findings should not be seen 
necessarily as negative but may imply that OCCs need to be aware of the influence of 
controller background. 

Participant response 

Respondents reported that they enjoyed participating in the simulation. One respondent 
(Male, 41-50 yrs old, 2 years experience in OCC) who participated in the domestic 
simulation commented that he felt he should be reaching an immediate solution, but in the 
absence of co-workers, was unable to solve the scenario. 

Management response 

Management in the five OCCs participating in the study commented very favourably on the 
use of the simulations. None of die OCCs conducted simulations that investigated 
decision-making outcomes of controllers prior to the study. Managers suggested that the 
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simulation method may provide OCCs with a valuable tool for selecting suitable staff, and 
as a means of identifying staff development and training opportunities. 

Data collection methods in the current study 

The use of simulation in the current study provided a suitable means of investigating 
decision-making of controllers in OCCs. However simulation alone would not have 
yielded the degree of richness of information that was collected. The necessity to interact 
with participants and observe their behaviour as they made decisions was a critical aspect 
which contributed to the success of the data gathering method necessary for the study. The 
simulation also provided a suitable context in which respondents could verbalise their 
thoughts. In contrast, other individual methods of data collection fail to capture the degree 
of data necessary for an in-depth investigation of decision-making. T The study emphasised 
that the use of simulation in conjunction with observation, interviews and think-aloud 
verbal protocol was the most appropriate combination of methods to study decision-making 
in OCCs. 

Conclusion 

The study represents a preliminary investigation of the application of several methods of 
data collection in OCCs. A conclusion drawn was that no one method was sufficient to 
investigate controller decision-making. However a multi-method approach using 
simulation together with observation, interviews and think-aloud protocol provided a most 
appropriate means. The full results of the study should make a substantial contribution to 
the research to gain further understanding of thought processes behind OCC decision- 
making. The simulation method has wide application and should lead to the development 
of an appropriate management tool to identify selection and training opportunities, and 
measure performance of controllers in OCCs. The current study strongly lends itself to the 
development of a computer-based simulation. 
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